Web Survey Bibliography
Relevance & Research question
Mobile phones are replacing the PC as key devices in social science data collection. In daily life, mobile phones are used for short interactions. Successful data collection strategies over mobile phones should therefore also be brief for respondents.
Questionnaires for attitude research are often very long. We argue that there is a trade-off to be made. Should questionnaires on mobile devices remain long, risking dropout, or should such questionnaires be split up (from here on called chunks) to optimize data quality?
Methods and data
We report on an experiment conducted in the probability-based LISS panel in the Netherlands, carried out in December 2015. We used a ‘within’ design of data chunking. Panelists who owned a mobile phone with Internet connection were randomly assigned to either:
a) The normal survey (about 20 min)
b) The same survey cut into three chunks, with each chunk offered after a week
c) The same survey cut into ten chunks, with each chuck offered every other day.
Results
First, we investigated the number of complete and incomplete responses and looked at indicators for data quality (straightlining, primacy effects, survey length). We find that more respondents are completing the questionnaire when it is offered in chunks (condition b, and especially c), but also that chunking results in more item missings. We find little evidence for effects on data quality.
Finally, we report on the differences we find in the factor structure when the questionnaire was split into chunks, or was completed in one go.
Added value
The idea of data chunking is not new. ‘Planned-missingness’ designs have been implemented in web surveys successfully in the past. This study is however the first to study data chunking in the setting of mobile phone surveys. We believe that more and more data will be collected using mobile phones (already 5-25% of all web surveys are taken on mobile phones), and that understanding how to design questionnaires for mobile phones is of vital importance to both survey researchers, market researchers, and anyone using such data for substantive reasons in the future.
Web survey bibliography - Toepoel, V. (27)
- Using experts’ consensus (the Delphi method) to evaluate weighting techniques in web surveys not...; 2017; Toepoel, V.; Emerson, H.
- Data chunking for mobile web: effects on data quality; 2017; Lugtig, P. J.; Toepoel, V.
- Mobile-only web survey respondents; 2016; Lugtig, P. J.; Toepoel, V.; Amin, A.
- Online Surveys are Mixed-Device Surveys. Issues Associated with the Use of Different (Mobile) Devices...; 2016; Toepoel, V.; Lugtig, P. J.
- Doing Surveys Online ; 2016; Toepoel, V.
- The Effects of Adding a Mobile-Compatible Design to the American Life Panel; 2015; Toepoel, V.; Lugtig, P. J.; Amin, A.
- Higher Item Nonresponse Rates Caused by Slider Scales in Web Surveys; 2015; Toepoel, V.; Funke, F.
- Coding Surveys on their Item Characteristics: Reliability Diagnostics; 2015; Bais, F.; Schouten, B.; Toepoel, V.
- Investigating Response Quality in Mobile and Desktop Surveys: A Comparison of Radio Buttons, Visual...; 2014; Toepoel, V.; Funke, F.
- Informing panel members about study results; 2014; Scherpenzeel, A., Toepoel, V.
- Mixed-devices in a probability based panel survey. Effects on survey measurement error; 2014; Toepoel, V., Lugtig, P. J.
- Panel Conditioning in Difficult Attitudinal Questions; 2013; Binswanger, J., Schunk, D., Toepoel, V.
- Mobile devices a way to recruit hard-to-reach groups? Results from a pilot study comparing desk top...; 2013; Toepoel, V., Lugtig, P. J.
- Effects of Incentives in Surveys; 2012; Toepoel, V.
- Building Your Own Online Panel Via E-Mail and Other Digital Media; 2012; Toepoel, V.
- Recruiting A Probability Sample For An Online Panel: Effects Of Contact Mode, Incentives, And Information...; 2012; Scherpenzeel, A., Toepoel, V.
- How Visual Design Affects the Interpretability of Survey Questions; 2011; Toepoel, V., Dillman, D. A.
- Panel Recruitment via Facebook; 2011; Toepoel, V.
- Nonparametric Tests of Panel Conditioning and Attrition Bias in Panel Surveys; 2011; Das, M., Toepoel, V., van Soest, A.
- Response Quantity, Response Quality, and Costs of Building an Online Panel via Social Contacts.; 2011; Toepoel, V.
- Can Verbal Instructions Counteract Visual Context Effects in Web Surveys?; 2011; Toepoel, V., Couper, M. P.
- Words, Numbers and Visual Heuristics in Web Surveys: Is There a Hierarchy of Importance?; 2009; Toepoel, V., Dillman, D. A.
- Pictures in Web Surveys; 2009; Toepoel, V., Couper, M. P.
- Panel Conditioning in Web Surveys: A Comparison between Trained and Fresh Respondents; 2009; Toepoel, V., Das, M., van Soest, A.
- Visual Heuristics and Answer Formats in Rating Scales; 2009; Toepoel, V. Dillman, D. A.
- Design effects in web surveys: comparing trained and fresh respondents; 2008; Toepoel, V., Das, M., van Soest, A.
- Can I use a panel? Panel conditioning and attrition bias in panel surveys; 2007; Das, M., Toepoel, V., van Soest, A.